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i ^TT.n OF Tff P f TWVENTION 
The present invention relates to computer graphics 
systems and more specifically to an adaptive pixel 
»ultisampler for processing image primitives to generate 
anti-aliased pixel data for display. 

narxftttom ^ M»P fiTIMMXPV OF THE TWVgWTIOW 
In a typical computer graphics display application, a 
person views an image generated by the display system and 
displayed on a screen, conventionally, the image is 
composed by an array of individual picture-cells ("pixels"). 
Pixels are the dots of color that define the image. Each of 
the pixels is defined by characteristics such as color and 
intensity. Traditionally, pixels are organized into a 
15 two-dimensional array or grid defined by raster lines. Each 
line is drawn by scan displaying the individual pixels along 
the line. An explanation of images composed of individual 
pixels is provided in a book roster graphics: Princ iples, 
anri Practice. 2nd Edition, Foley, van Dam, Feiner ft Hughes, 
20 (Reprinted in 1991) (1990), by Addison-Wesley Publishing 
Company, see the section beginning on page 9. 

In practice, rather than defining the characteristics 
of each pixel, display systems define an image from a 
collection of primitive graphic shapes such as polygons. 
25 The polygons are numerically represented in terms of size, 
color, location and other characteristics. To accomplish an 
image, the polygon representations are processed by an image 
generator producing pixel data signals by a process called 
polygon rendering. In effect, polygon rendering involves 
30 sampling a polygon to calculate how the polygon influences 
the pixels. 

Display systems typically generate each pixel data 
signal by sampling several locations within the pixel of the 
polygon- This sampling technique is used to reduce 
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undesirable visual artifacts in the displayed image. These 
visual artifacts, commonly referred to as aliasing, often 
appear as jagged lines at the polygon boundaries. 

Conventionally, systems that use multiple sampling 
5 locations for each pixel treat pixels as square or 

rectangular regions in the image plane which are subdivided 
by a grid or array of sub-pixel points. Typically, the 
regions defined by the sub-pixel points abut their neighbors 
uniformly in both directions and cover the image plane 

10 exactly once. Each point is then considered with regard to 
each influencing scene detail (polygon) to generate a value 
for each sub-pixel point. Generally, the final color for a 
pixel is calculated by combining the sub-pixel values for 
that pixel. However, some display systems compute the final 

15 pixel color from a weighted average of the image information 
in a two-by-two set of pixels, that is, from four pixels 
worth of sub-pixel values. 

The number of sub-pixels used to generate the final 
color for each pixel has a direct impact on the cost of the 

20 display system. Display systems typically store data for 
each sub-pixel in the pixel. Thus, pixel data memory 
(commonly referred to as "frame buffer" memory) must be 
allocated for every sub-pixel in every pixel. For exampLe, 
a graphics system that uses 16 sub-pixels per pixel and has 

25 a display with a 1024-by-1024 array of pixels would use over 
16 million frame buffer locations for the sub-pixel data, 
Due to the relatively high cost of frame buffer memory, i% 
need exists for a computer graphics display system that 
produces anti-aliased images but uses less frame buffer 

30 memory. 

The present invention provides a novel memory 
allocation scheme and a novel sub-pixel sampling technique 
to reduce memory requirements and provide high quality anti- 
aliased images. The memory allocation scheme involves 
3 5 storing pixel data in the frame buffer on a per-polygon, 

per-pixel basis rather than storing data for each sub-pi:cel 
as discussed above. The sampling technique involves 
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generating pixel data sub-pixels using an interlocking pixel 

pattern. , . . „ 

Pixel data is stored on a per-polygon, per-pixel basis 
in the frame buffer by separately storing each polygon that 
5 influences a pixel in memory locations allocated for that 
pixel. For example, if a pixel is influenced by two 
polygons, the frame buffer Will contain two sets of data for 
the pixel, i.e., one set for each polygon. If mother pixel 
is influenced by only one polygon, the frame buffer will 
10 contain only one set of data for that pixel. Thus, for each 
pixel that is influenced by at least one polygon, one set of 
polygon data is stored in the frame buffer for each polygon 
that influences the pixel. 

Unlike conventional sub-pixel display systems, the 
15 polygons are not blended with other polygons as they arn 
added to the frame buffer. Instead, each polygon that 
influences the pixel is stored in the f»~ buffer. The 
polygons are blended at a later stage in the pixel data 
generation process. 
2 o The per-polygon, per-pixel frame buffer scheme uses 

less memory because memory is allocated for each polygon 
that influences a pixel rather than for each sub-pixel 
associated with the pixel. In practice, each pixel is 
influenced by relatively few polygons. Thus, less memory is 
25 needed to Store the polygon information than would be needed 
to store the sub-pixel information. 

Turning now to the interlocking pixel pattern mentioned 
above, the pixel pattern is defined by sub-pixel areas, each 
of which is associated with a sub-pixel. The sub-pixel 
30 areas are arrayed so that some of the sub-pixel areas lie 
outside the area conventionally defined for the pixel and 
lie within a "hole" defined in the adjacent pixels' pattern. 
The arrayed sub-pixel areas and the holes are defined so 
that each part of the image plane is covered by one and only 
35 one sub-pixel area. Thus, the sub-pixel areas form a 

contiguous, non-overlapping tiling of the pixel image plane 
yet provide a wider sampling area for each pixel. 
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The interlocking pixel technique can increase the 
quality of the displayed image by reducing aliasing 
associated with image problems such as brightness slope 
discontinuities- Moreover, this technique does not 
5 significantly increase system memory requirements and can be 
implemented with relatively minimal impact on image 
processing speed. 



BRIEF DESCRIPTION OF THE DRAWINGS 
In the drawings which constitute a part of this 
10 specification, an exemplary embodiment exhibiting various; 
objectives and features hereof is set forth, specifically: 

FIGURE l is a block diagram illustrating one embodiment 
of a computer graphics system constructed according to the 
present invention ; 
15 FIGURE 2 is a flowchart of a pixel data generating 

process executed by the system of FIGURE l; 

FIGURE 3 is a graphic representation of a three- 
dimensional view frustum and a frame buffer organization 
illustrating operations as treated herein; 
20 FIGURE 4 is a graphic representation illustrating a 

sub-pixel pattern used in one embodiment of the present 
invention; 

FIGURE 5 is a graphic representation illustrating the 
interlocking nature of the sub-pixel pattern of FIGURE 4; 
25 FIGURE 6 is a graphic representation of memory 

organization in one embodiment of a frame buffer as treated 
herein; 

FIGURE 7 is a side view of a pixel frustum in a model 
space as treated herein; 
30 FIGURE 6 is a sectional view from the eye-point of 

FIGURE 7 taken along line 8-8 of FIGURE 7; 

FIGURES 9A and 9B are flowcharts of a pixel shade 
generating process executed by the system of FIGURE 1; 
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FIGURES 10A and 10B are block diagram* illustrating two 
embodiments of a pixel shader according to the present 

invention; and 

FIGURES HA and 11B are flowcharts of pixel shading 
5 processes executed by the embodiments of FIGURES 10A and 
10B , respectively - 

nKSCRTPTlON ™w TT.LUSTRflTTVyi EPBODIflEHT 
As required, a detailed illustrative embodiment of the 
present invention is disclosed herein. However, computer 
10 graphics systems, component operating structures, frame 
buffer memories, polygon rendering techniques, pixel 
processing techniques, sampling techniques, sorting 
techniques and blending techniques as well as other elects 
utilized in accordance with the present invention may bn 
15 embodied in a wide variety of forms, some of which may be 
quite different from those of the disclosed embodiment. 
Consequently, the specific structural and functional derails 
disclosed herein are merely representative; yet in that 
regard, they are deemed to afford the best embodiment for 
20 purposes of disclosure and to provide a basis for the claims 
herein which define the scope of the present invention. 

Referring initially to FIGURE 1, one embodiment of a 
computer graphics system conducted in accordance with the 
present invention is shown. A multisampler 24 (top) 
25 processes polygons stored in a polygon memory 22 (bottom 
left), initially, the multisampler 2 4 processes the 
polygons to determine which pixels are influenced by the 
polygons. This process uses an interlocking sub-pixel 
pattern P (FIGURE 4) . After determining which pixels the 
3 0 polygons influence, the multisampler 24 stores data 

associated with the polygons in a frame buffer 26 (bottom 
center) on a per-polygon, per-pixel basis. In other words, 
if a pixel is influenced by at least one polygon, one packet 
of data will be stored in the frame buffer 26 for every 
3S polygon that influences the pixel. The multisampler 2 <l 

processes these polygon data packets to generate color and 
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intensity data for each pixel. This data is sent to a 
display device 20 (bottom right) to generate images on a 
display screen (not shown) . 

Referring to FIGURES 1 and 2, the basic operation ol 1 
5 the disclosed embodiment will be discussed in more detail. 
FIGURE 2 illustrates the process of generating a frame of 
pixel data for display. Initially, a scene to be displayed 
on the display device 20 (FIGURE 1) is defined by a 
collection of polygons in a three-dimensional model space 

10 (not shown) . These polygons (actually, numerical 

representations of the polygons) are stored in the polygon 
memory 22 (FIGURE 1) « When the polygons for a given image 
frame are to be displayed, the multisampler 24 processes the 
polygons to generate pixel data beginning at a block 170 

15 (FIGURE 2, top) . 

At a block 172 (FIGURE 2), the sampling engine 26 
(FIGURE 1, top left) retrieves a polygon from the polygon 
memory 22. At a block 174, using the interlocking sub-pixel 
pattern P (FIGURE 4), the sampling engine 28 determines 

20 which pixel sub-*pixels are influenced by the polygon. In 
the pattern P depicted in FIGURE 4, four sub-pixels (e.g., 
sub-pixels 56) of the total sixteen sub-pixels (e*g w sub- 
pixels 56 and 58) are displaced laterally into screen areas 
normally occupied by the adjacent pixels. In turn, the 

25 pattern leaves a hole 60 in itself that each of the adjacent 
pixels can use, FIGURE 5 shows that the pattern interlocks 
and tessellates properly across the image plane, The sub- 
pixels 56a and 56b from pixels 62a and 62b r respectively, 
use part of the hole 60 in pixel 62c. Thus, every portion 

3 0 of the image plane belongs to exactly one pixel. In 

practice, single sampling points would be used to sample 
within the sub-pixel areas depicted in FIGURES 4 and 5. 

The sampling engine 28 generates a sub-pixel mask which 
identifies the sub-pixels influenced by the polygon. In 

35 addition, the sampling engine 28 generates color and 

transparency information for the pixel by interpolating 
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polygon value, at the area of the polygon that influence,. 

the pixel. . 

For each pixel influenced by the polygon, the sampling 
engine 28 sends the sub-pixel mask and polygon information 
5 to the next stage of the process. The following discussion 
of blocks 176 through 186 describes processing the mask ana 
polygon information for a single pixel. 

At a block 176, the multisampler 24 retrieves the 
polygon information of any previously processed polygons 
10 that influence the pixel currently being processed. This 
polygon information is stored in the frame buffer 26 on . 
per-polygon, per-pixel basis. The per-polygon, per-pixel 
relationships between pixels, polygons and frame buffer 
memory are illustrated in FIGURE 3. 
15 In the upper half of FIGURE 3, a three-dimensional 

model space 4 0 defined by the view from an eye-point E 
through a display screen 44 is shown. Polygons 46* and 46b 
are defined in the model space 40. Rays 50a, 50b and 50c 
represent the line of sight from eye-point E through display 
20 screen pixels 48a, 48b and 48c, respectively. As 

represented by the intersection of the rays 50a and 50b with 
the polygons 46a and 46b, pixel 4aa is influenced by 
polygons 46a and 46b and pixel 48b is influenced by polygon 
46a. Pixel 48c is not influenced by either polygon. 
25 The bottom half of FIGURE 3 illustrates how data is 

stored in the frame buffer 26 for this example. Polygon 
information for polygons 46a and 46b is stored in the. frame 
buffer 26 and associated with pixel 48a. Similarly, polygon 
information for polygon 46a is stored in the frame buffer 26 
30 and associated with pixel 48b. No polygon information is 
stored in the frame buffer for pixel 48c because it is not 
influenced by a polygon. 

Referring again to FIGURE 1, the occupation solver 32 
(top right) resolves hidden surface problems that arise when 
35 multiple polygons influence a single pixel (FIGURE 2, block 
178) . Hidden surfaces are accounted for as each previously 
processed polygon is read from the frame buffer 26 and 
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stored in a polygon buffer 30 (FIGURE l, center) . The 
occultation solver 32 computes an analytic description for 
the edge of intersection of each new polygon and each 
previously processed polygon that influence the pixel. A 
5 sub-pixel matrix corresponding to this implied edge is 

applied to the new and previous polygon to determine which 
sub-pixels are influenced by which polygons. After this 
procedure is performed for all the previous polygons, the 
surviving sub-pixels for the new polygon are saved with its 

10 shade data* Special processing properly handles the 
interaction of opaque and transparent polygons. 

At a block 180, the depth sorter 34 (FIGURE 1, top 
middle) sorts the polygons stored in the polygon buffer 3 0 
that influence the pixel. The polygons are sorted according 

15 to the polygons' pixel-center depth values. As new polygons 
are added, they are inserted into the prior list in the 
appropriate location. This depth ordering is later used to 
apply transparency effects properly. 

The pixel shader 36 (FIGURE 1, right middle) computes a 

20 final shade for each pixel by processing the polygons th*t 
influence the pixel in depth order (FIGURE 2, block 182) . 
It uses the sub-pixel, transparency and color information 
for each polygon to properly combine both color and 
transparency effects . 

25 After the final shade is calculated, surviving polygons 

are written back to the frame buffer 26 in depth order 
(FIGURE 2, block 184). As discussed above, these polygons 
are stored on a per-polygon, per-pixel basis. At a block 
186, the final pixel shade is stored in the frame buffer 26. 

30 At a block 188, if any more polygons need to be 

processed for the current frame, the above process is 
repeated for the next polygon. Once all the polygons have 
been processed for a frame, the final pixel shades are road 
from the frame buffer 26 and sent to the display device :io 

35 which uses the final shade data to drive its illuminant 
pixels (block 190) . Processing for this frame then 
terminates at a block 19 2. 
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To further an understanding of the disclosed 
■ embodiment, the components of FIGURE 1 will be treated in 
detail. Referring again to FIGURE 3, the images seen on the 
display screen 44 at an eye-point E are formed from the 
5 polygons in the model space 40. In practice, a polygon' o 
characteristics such as shape, location in the three- 
dimensional space, color and opacity are represented by one 
or »iore data values. Prior to processing, the data values 
for the polygons (referred to herein simply as polygons) are 
L o stored in the polygon memory 22 (FIGURE 1) . 

To generate images on the display screen 44, the 
polygons defined in three-dimensional space are mapped to 
pixels (e.g., pixel 48A, shown greatly exaggerated) in the 
two-dimensional screen. The intersection of the pixel 
15 frustum (represented in this simplified example by rays, 
e.g., ray 50a) with a polygon (e.g., polygon 46a) defines 
which portion of the polygon is mapped to the pixel. Tc 
accomplish the mapping of a polygon's characteristics tc a 
pixel, the multisampler 24 (FIGURE i) retrieves the polygon 
20 (i e., the numerical representation of the polygon) from the 
polygon memory 22 over the line 54 and, in effect, samples 
the polygon at sub-pixel locations corresponding to that 

pixel. . 

For each pixel that a polygon influences, the sampling 
25 engine 28 ( FIGURE 1) determines Which sub-pixels lie inside 
the polygon using the polygon's edge descriptions. Each 
edge is characterized by its slope and a slope index is 
computed once per edge. The perpendicular distance fro* the 
pixel to the edge is computed once par-pixel, per-edge, then 

30 quantized and combined with the slope code to form a full 
sub-pixel pattern table address. This table is stored in a 
memory (not shown) . Pattern bits for each of the three 
edges are read from the memory and logically "anded" 
together to yield the final sub-pixel mask. This masJc 

35 defines the shape of the polygon within this pixel. The 

details of the method set forth above for determining vhich 
sub-pixels are influenced by a polygon using the polygon's 



05/08/2003 12:00 FAX 404 873 8501 



ARNALL GOLDEN & GREGORY 



©046 



WO 97/41536 PCT/US97/07-I47 

10 

edge descriptions is well known in the computer graphics art 
and will not be discussed further here- For example, see 
U.S. Patent No. 4,918,626 (Computer Graphics Priority System 
With Antialiasing, Watkins, et al.)- 
5 The sampling engine 28 uses a pixel sampling pattern 

with arrayed sub-pixels as discussed above in conjunction 
with FIGURES 4 and 5. This sampling pattern is used to 
reduce aliasing in the displayed image, in particular, 
display systems that generate the final pixel color using 

10 unweighted single-pixel sampling often produce images where 
nearly horizontal or vertical edges tend to look "ropy 11 or 
scalloped* The present invention is based, in part, on the 
realization that this visual effect is caused by the manner 
in which the visual processing of the human eye and brain 

15 handles brightness slope discontinuities caused by the 

unweighted single-pixel sampling method. These brightness 
slope discontinuities result from a polygon having no effect 
on a pixel when it is not within the pixel pattern (even if 
the polygon is immediately adjacent to the pixel) and having 

20 maximum effect on the pixel when it covers the pixel. Thus, 
the polygon's effect on the pixel varies from zero to 
maximum in the space of one pixel. 

The pixel pattern P of FIGURE 4 reduces the effect 
these polygon discontinuities have on a pixel* First, the 

25 polygon influences the pixel over a wider area. Second, the 
pattern sub-pixel density varies across the pattern. As a 
result, the polygon influences the pixel sooner and has more 
effect on the pixel the closer the polygon is to the pix<2l. 
Consider, for example, a horizonal polygon edge as it 

3 0 approaches the bottom of the pixel pattern of FIGURE 4. 

Initially, the edge is immediately below the bottom of the 
pixel, takes no sub-pixels and has no effect on the display 
video. As the edge moves upward, the polygon influences one 
sub-pixel after the edge traverses a distance of 

35 approximately one half of a pixel (as measured by a 

conventional square pixel) . As the edge traverses the next 
one half pixel width, the polygon influences more sub- 
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pixels. Finally, as the edge traverses the lest one hall: 
pixel width, the polygon affects the last sub-pixel. Thus, 
the polygon'- effect on the pixel "ramps up" as the polygon 
edge approaches and eventually occults the pixel and ramps 
down as the polygon edge moves past the pixel. Moreover, 
the polygon influences the pixel over a distance of more 
than one and a half pixels. By softening the 
discontinuities in this manner, the ropy or scalloped edje 
effects discussed above are greatly reduced. 

The disclosed embodiment achieves these results without 
the architectural complexity of a multi-pixel shaped filter 
and without using more than one pixel of sub-pixels. 
Typically, a multi-pixel shaped filter computes the final 
pixel color from a weighted average of the scene detail in a 
two-by-two set of pixels. However, generating the weighted 
averages in these systems typically requires relatively high 
computational power and appropriate data cross-links between 
adjacent pixels. In contrast, pixel-to-pixel cross-linKs 
are not required in the disclosed embodiment because it uses 
a single pixel of sub-pixels. Moreover, because the sub- 
pixels are interlocked, a wider area is sampled without the 
use of additional sub-pixels for each pixel. 

The implementation cost of the disclosed embodiment is 
quite low. The active distance of the filter is a little 
25 larger, so the scan conversion process needs to overscan 

polygons by a fraction of a pixel. However, this capability 
is already needed to support some graphics Application 
programmer Interface modes. In addition, a few more 
distance codes may need to be in the pattern memory. 
3 0 However, the downstream processes are unaffected because the 
final filtering process still uses equal, or "flat," 
weighting- In sum, this sampling technique provides 
improved visual displays with relatively low additional 
cost. 

3 5 The sampling engine 28 generates the color and 

transparency data for the pixel by scan converting the 
polygons retrieved from the polygon memory 22. Basically, 
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data values for the pixel are generated by calculating the 
respective values at the location on the polygon that 
corresponds to the pixel. Typically, this involves 
interpolating a value using the values defined at the 
5 polygon's vertices- The scan conversion process is well 
known in the computer graphics system art and is discussed, 
for example, in the books Principles of Interactive computer 
Graphin g r 2nd Edition, Newman and Sproull, McGraw-Hill Book 
Company, 1979, and Computer Graphics : Principles and 

10 Practice . 2nd Edition, Foley, van Dam, Feiner & Hughes, 
Addison-Wesley Publishing Company, 1991. 

After the sampling engine 28 calculates the sub-pixel 
mask and polygon information using the techniques described 
above, it sends this data to the next stage of the 

15 mult is* ampler 24. The multisampler 24 stores the mask and 
polygon information in the frame buffer 26. Accordingly, 
the organization and operation of the frame buffer 2 6 will 
now be treated, 

A portion of the frame buffer 2 6 is allocated to store 

20 pixel-specific information associated with the active video 
screen. This information includes the pixel video color 
(which is typically double buffered) , stencil and other 
combining information. In addition, heap management 
information for the polygons influencing the pixels is 

25 stored here. Typically, this portion of the frame buffer is 
addressed by pixel row and column values, with appropriate 
window offsets and other addressing information. The 
details of these aspects of the frame buffer are well known 
in the computer graphics art and will not be discussed here. 

3 0 A second portion of the frame buffer 2 6 stores the -.uask 

and polygon information generated by the sampling engine 28 
as discussed above. Typically, this data includes the 
color, opacity, depth value and depth gradients for the 
polygon and a sub-pixel use mask which describes which pixel 

35 sub-pixels are influenced by the polygon. This information 
is stored in the frame buffer 26 on a per-polygon, per-plxel 
basis. In other words, each pixel will have one set of the 
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above information stored in the frame buffer 26 for every 
polygon that influences that pixel. 

This portion of the frame buffer memory is managed <is a 
heap 64 (FIGURE 1) . Each new polygon discovered within * 
5 pixel is given an increment of memory in which to store Lts 
shade and occultation information and pointer linXs that are 
used to connect the various polygons that influence each 
pixel. These pointer links allow prior polygons for a given 
pixel to be quickly retrieved as the effects of a new 
10 polygon are added. The heap portion of the memory is 
managed in polygon-per-pixel data packets. 

FIGURE 6 graphically illustrates data packets in an 
exemplary embodiment of a frame buffer that has a portion of 
it organized as a heap. Tor each pixel influenced by a 
15 polygon (designated PIXEL(l) through PIXEL(N) J , one or more 
data packets (e.g., data packets 66b and 66c) are stored in 
the frame buffer 26. One data packet is stored for each 
polygon that influences a given pixel. For example, data 
packet 66b contains the information for one polygon that; 
20 influences PIXEL ( 1) . Data packet 66c contains the 

information for another polygon that influences PIXEL(1} . 
As discussed above, each data packet typically contains 
polygon information (e.g., box 68b), a sub-pixel mask (e.g., 
mask 70b). and a pointer to the next data packet (e.g., 
25 pointer 72b). As represented by the line 74, the pointer 
contains the frame buffer memory address of the next data 
packet associated with the same pixel. For example, pointer 
72b would contain the address for data packet 66c. 

The heap data packets and the associated pointers are 
30 managed by a heap manager 73 (FIGURE 1, center). The heap 
manager 73 initializes the heap pointers at the start of 
every frame and allocates new data packets as needed during 
the rendering process. The heap pointers are valid up to 
the maximum number of polygons within the pixel. This 
35 maximum is initialed at the start of each frame. 

In one embodiment, the heap manager 73 does not attempt 
to reclaim heap space during frame processing even though 
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heap space is vacated whenever a new polygon completely 
obscures a prior one within a pixel. Instead, the pixel- 
specific portion of the memory (discussed above) contains 
two values which indicate how many polygons are currently 
5 active within the pixel and what the maximum number of 
active polygons has ever been during the current frame. 
When the number of currently active polygons is less than 
the maximum number of active polygons, heap space has been 
allocated, used, and abandoned by prior polygons. In this 

10 case, the embedded heap pointers are still valid because the 
heap location of an obscured polygon remains linked to the 
pixel for the remainder of the frame. Thus, if another 
polygon needs to be added for this pixel , the memory is 
already allocated and linked in. Consequently, a modest 

15 level of heap re-use is provided without the complexity of 
heap reclamation. 

The heap manager 73 provides for two types of heap 
overflow. The first type involves overflow of the internal 
buffers. Since the raulti sampler 24 reads all prior polygons 

20 associated with the pixel before it begins shade 

computations, internal buffers are provided to store the 
polygon data packets. When a pixel contains too many 
polygons, the system uses the allowable number of polygons, 
from closest to farthest (based on the depth sort order) , 

25 and drops the farthest one. These polygons will be dropped 
one at a time because the system does not output an 
excessive number of polygons back to the frame buffer 26. 

The second type of overflow involves overflow of the 
heap space. If the system runs out of heap space, 

30 subsequent pixel processing continues and, for each pixeL 
that needs additional space, the farthest polygon gets 
dropped on output. This might not leave the pixel properly 
shaded, depending on whether any other polygons were dropped 
within this pixel. Nevertheless, the possibility of heap 

3 5 overflow is lessened somewhat because new polygons often 
make their own heap space by completely occulting a prior 
polygon and freeing up its data packet. 
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I„ one embodiment of the present invention, a portion 
• of the frame buffer memory is pre-allocated to pixel, ttot: 
are influenced by only a few polygons. In FIGURE 6 (left) . 
the memory for data packet 66a is pre-allocated and is not 
part of the heap. Thus, the information (e.g., box 6Ba and 
mask 70a) for the first polygon that influences the pi»- 
vill always be stored here. This technique increases th» 
rate at which these data packets are accessed because no 
neap processing is performed when the packets are stored or 
, retrieved. Memory can be pre-allocated for as many pixeLs 
as needed. Thus, the «*" in PIXEL(N) represents the numoer 
of pixels that have memory pre-allocated for them. In 
addition, memory may be pre-allocated for more than one data 
packet for each pixel if needed. 
15 in another embodiment, a portion of the available frame 

buffer memory is pre-allocated to store the first polygon 
data packet for each pixel. Here, it is assumed that every 
portion of the screen will be influenced by at least one 
polygon. FIGURE 6 illustrates this organization where the 
2 0 »N» in PIXEL (N) represents the total number of pixels. As 
above, this technique increases the rate at which these 
packets are accessed because no heap processing is 
performed . 

The per-polygon, per-pixel memory allocation schema 
25 discussed above can reduce frame buffer memory reguiremnnts 
by 50% to 75% over frame buffer memory allocation schemes 
that allocate memory for every sub-pixel in every pixel . 
This memory savings is realized because memory is allocated 
for each polygon that influences the pixel rather than for 
3 0 each pixel sub-pixel. Since typical multisampling 

applications use 16 sub-pixels per pixel but average onLy a 
few polygons per pixel, less memory is needed to store the 
polygon information than the sub-pixel information. 
Except for specially constructed, deliberately 
35 pathological test images, it has proven very difficult to 
overrun a local buffer maximum of sixteen polygons. It has 
also proven very difficult to exceed a heap allocation 
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corresponding to an average of four polygons per pixel. For 
most imagery, an average of three polygons per pixel is 
normal- Moreover , high quality images are typically 
produced using an average of less than two polygons per 
5 pixel. 

When the heap limits are exceeded, the visual effect of 
dropping the farthest polygon is not necessarily 
catastrophic. Typically, the observed image degradation has 
been fairly subtle. In any event, if a user needs a system 
10 where the heap is essentially never overrun, the system can 
be configured with storage several times over the baseline 
limits . 

Referring again to FIGURE 1, the remaining components 
of the multisampler 24 will be discussed. These components 
15 process the polygons and masks generated by the sampling 

engine and stored in the frame buffer. As discussed below, 
this process involves resolving polygon occultations r 
sorting the polygons and blending the polygons to generate 
pixel data. 

20 The occultation solver 32 (FIGURE 1) compares the 

polygons that influence a given pixel to determine whether 
any of these polygons occult each other. As FIGURE 3 
illustrates, one polygon (e.g., polygon 46a) may be in front 
of another polygon (e.g., polygon 4 6b) in the model space. 

25 When this occurs, some of the sub-pixels of the farthest 
polygon may be occulted by the nearest polygon. 

As each new polygon is output by the sampling engine 
28, the occultation solver 32 compares the new polygon with 
any previously processed polygons (the "prior polygons") 

30 that influence the pixel being processed. This comparison 
is done one polygon at a time as the prior polygons are .read 
from frame buffer memory into local buffers. When a 
polygon's sub-pixel is occulted by another polygon, the 
occultation solver 32 modifies the occulted polygon's 

35 coverage mask to reflect that this polygon no longer 
influences that sub-pixel. 
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The occupation solver 32 compares the polygon sub- 
pixels as follows. The depth at a ^.^ U 
the depth at the pixel center, plus the relative 
the sub-pixel from the pixel center (in X and Y) , times the 

5 respective gradient values (in x and Y). This is 

reformulated to reduce the amount of computation required to 
comp are the sub-pixel depths. Initially, the new polygon's 
and the prior polygon's pixel-center depth values and depth 
gradient values are subtracted from each other. This 

XO calculation only needs to be done once per pixel. Then, the 
depth test at each sub-pixel is simply the depth difference 
at the pixel center, plus the gradient differences times the 

suto-Dixel offsets: 

Ds . (Dn-Dp> + <DXn-DX P )*Xs + (DYn-DYp) *Ys EQUATION 1 
15 Where Ds. Dn and Dp denote the depth values of the sub- 

pixel, the nev polygon and the prior polygon, respectively. 
DXn, DYn, DXp and DYp denote the »X'« and "Y" direction depth 
gradient values for the new polygon and prior polygon, 
respectively. Xs and » denote the sub-pixel offsets in the 
20 »X" and "Y» directions, respectfully. Since the depth value 
D is 1/8 and hence gets smaller with range, if Ds is 
positive, the new polygon is closer and should be given 
visual precedence over the prior polygon. 

Further efficiencies are obtained based on the 
25 realization that the above equation is in the form of .i Ixn. 
equation: AX+BY+C = 0, where A = DXn-DXp, B = DYn-DYp and C 
= Dn-Dp - Specifically, the line equation is the line of 
intersection of the two polygons as defined by their d«*pth 
and gradient information. On one side of this line, the new 
30 polygon is visible. On the other side of the line, the 

prior polygon is visible. A and B are the components of a 
vector that points normal to the edge, towards the new 
polygon's side of the three-dimensional space. When the 
equation is normali.ed by dividing A, B and C by the square 
3 5 root Of A ! +B*, C is equal to the perpendicular distance, in 
pixel units, from the pixel center to the edge. Thus, A and 
B can be used to develop an edge slope code, combined with 
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the perpendicular distance and used to look up the sub-pixel 
pattern corresponding to the edge of intersection- This 
process is identical to the one used in the sampling engine 
28 and can, in fact, make use of the same pattern table 
5 information- See, for example, U.S. Pat. No, 4,918,626, 
cited above, 

The pattern retrieved from the table has sub-pixels set 
where the new polygon prevails- The pattern is logically 
"anded" with the new polygon's sub-pixels to define the 

10 condition where the new polygon exists and the new polygon 
prevails. The inverse of the "new polygon exists and 
prevails" pattern is "anded" with the prior polygon to 
eliminate any sub-pixels of the prior polygon that are 
claimed by the new polygon. In other words, the prior 

15 polygon's sub-pixels that are occulted by the new polygon's 
sub-pixels are removed from the prior polygon's sub-pixe] 
mask. 

Next, the inverse of the pattern retrieved from the 
table is 11 anded " with the prior polygon to define the 

2 0 condition where the prior polygon exists and the prior 

polygon prevails. The inverse of the "prior polygon exists 
and prevails" pattern is "anded 11 with the new polygon to 
eliminate any sub-pixels that are claimed by the prior 
polygon. Thus, the new polygon's sub-pixels that are 
25 occulted by the prior polygon's sub-pixels are removed from 
the new polygon's sub-pixel mask. 

In the process above, an opaque polygon occults 
everything behind it by erasing the conflicting sub-pixel 
bits for the polygons behind it. In contrast, a transparent 

3 0 polygon does not occult anything. Thus, the new polygon's 

sub-pixel mask is modified as described above only if the 
prior polygon is opaque. Similarly, the prior polygon's 
sub-pixel mask is modified only if the new polygon is 
opaque * 

35 The procedure set forth above provides an efficient 

method of eliminating hidden surfaces, in comparison, it 
has been proposed to eliminate hidden surfaces by computing 
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the depth values for e a ch sub-pixel of the new and prior 
' polygon*, then keeping the closest sub-pixels and erasing 
the farthest ones. The depth and depth-gradient values 
associated with the polygon data make this approach 
, straightforward. However, this approach typically requires 
substantial processing to perform the necessary 
calculations, in contrast, the disclosed procedure requires 
less processing by utilizing the shortcuts discussed above. 
After the occupation solver 32 eliminates hidden 
3 polygon surfaces, the depth sorter 34 (FIGURE 1) sorts the 
polygons according to the relative depth of the polygons in 
the model space. The depth sorter 34 sorts the polygon* as 
each new polygon is sent from the sampling engine 28. As 
each prior polygon is read from the frame buffer 26, the, 
5 prior polygon's pixel-center depth is compared with the new 
polygon's pixel-center depth value. For example, referring 
to FIGURE 7, a side view of a pixel f return defined by the 
view from eye-point E through a pixel 82 along lines so is 
shown, in the center of pixel 82 (designated by line 8f>) , 
o polygon 84a is in front of polygon 84b. Thus, polygon 84a 
would be placed ahead of polygon 84b in the list. By the 
time all prior polygons have been read into temporary 
buffers, the proper insertion point for the new polygon has 
been determined. When the surviving polygons are written 
>S back to the frame buffer 26, they will be arranged m this 
new order. Thus, the prior polygons that accumulate for a 
pixel are always resident in the frame buffer 26 in their 
pixel-level depth sort order. New polygons simply are 
merged into the proper place in the list. 
30 This procedure provides an efficient method of sorting 

polygons. In comparison, a rigorous depth ordering solution 
would require ordering on a per-sample basis. For example, 
all the polygons using sample 5 would be ordered into a 
depth list unique to sample 5. This approach leads to a 
35 pixel shader process complexity that grows non-linearly with 
the number of polygons in a pixel. In contrast, the 
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procedure set forth above requires only a pixel-level d*pth 
sort and yields a linear pixel shader process time. 

After the polygons for a pixel have been processed by 
the occultation solver 32 and the depth sorter 34, the pixel 
5 shader 36 (FIGURE 1) processes the polygon information to 
generate the final shade for each pixel. As FIGURE 8 
illustrates, a pixel may be influenced by more than one 
polygon. For example, if a semi-transparent polygon (e.g., 
polygon 84a) is in front of another polygon (e.g., polygon 
10 84b), both polygons will contribute to the pixel. 

Similarly, if two opaque polygons each occupy a portion 
(e.g., areas 92 and 94) of the pixel, the effects of both 
polygons should be taken into account when calculating the 
pixel shade. 

15 The pixel shader 36 accumulates shade and transparency 

effects in depth-sort order. One embodiment of the pixel 
shader 3 6 is depicted in FIGURE 10A. A cumulative 
transparency calculator 130 develops a cumulative 
transparency for each level in the sorted polygon list using 

20 sub-pixel use masks 132 and cumulative use sub-pixel masks 
134 generated by a sub-pixel allocation calculator 136. A 
cumulative shade calculator 138 uses the cumulative 
transparency values to calculate how much of the current 
polygon should be affected by prior transparency and how 

25 much should show through unaltered. The values are computed 
for each new level from the previous level's values. 

The cumulative mask is the logical "or" of the 
prior-level cumulative mask with the current polygon sub- 
pixel mask. Table 1 illustrates sub-pixel use masks and 

3 0 cumulative use masks for five levels (polygon 0 - polygon 
4) . The sub-pixel allocation calculator 136 also generates 
various mask bit counts and polygon level identifiers 
(designated A - E in TABLE 1) that the pixel shader uses to 
generate the final pixel shade. "A" designates the number 

35 of active sub-pixels (as represented by the cumulative sub- 
pixel use mask) at the current level. "B 11 designates th<2 
nearest prior level where the cumulative use mask has at 
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least one sub-pixel bit set which is also set in the current 
" level. »C» designate** the number of sub-pixels used by the 
current polygon that are not used by any prior polygons. 
« D «. designates the number of sub-pixels used by both the 
5 current polygon and at least one prior polygon. -E- 
designates the number of sub-pixels used by any prior 
polygons that are not used by the current polygon. 

T^BLE 1 

SUB-PIXEL USE MASKS CUMULATIVE USE MASKS A B C 0 B 
in r,olv4- 1111110000000011 llllllllllXlllH 16 3 O 8 0 

P oly2 0000001111111100 1110001111111100 i 

Eolyi: ooooooiiooooiioo i 1100 ?i"}}"J?2 x l " . - I 

polyo: moooooiiiioooo 1110000011110000 7 

15 Starting at a block 100, FIGURE HA illustrates how the 

pixel shader 36 of FIGURE 10A generates a shade for a pixel. 
Basically, the pixel shader 36 calculates the use masks and 
cumulative transparency and uses these values to generate 
the final pixel shade. At a block 102. the pixel shader 36 
20 retrieves the data for the current polygon from the polygon 
buffer over a line i«o (FIGURE IDA) . This data include* the 
sub-pixel mask and polygon information generated by the 
sampling engine 28 (FIGURE 1) as discussed above. 

At a block 104, the pixel shader retrieves the prior 
25 sub-pixel masks 132 and cumulative use masks 134 over lines 
142 and 144, respectively. These masks were generated when 
the previous polygons in the list were processed. 

At blocks 106, 108 and 110, respectively, the sub-pixel 
allocation calculator 136 calculates and stores the values 
30 for »C» "E" and »D« (TABLE 1) for the current level. In 

addition, the sub-pixel allocation calculator 136 calculates 
and stores the new cumulative use mask, the current 
polygon's sub-pixel mask and the value for "A" (block 112). 
At a block 114, the pixel shader retrieves the 
35 cumulative transparency for the prior level and calculates 
and stores the cumulative transparency for the current 
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level. The current level cumulative transparency is a 
weighted average of all prior transparent and opaque 
polygons. For the first polygon in the list, the cumulative 
transparency is simply; 
5 CT(0) *= 1 - op(0) EQUATION 2 

Where CT{0) and op(0) are the cumulative transparency and 
opacity of the polygon at level 0, respectively. The 
cumulative transparency calculator 13 o accesses the currant 
polygon's opacity data over a line 146 (FIGURE 10A) - 

10 For subsequent polygons in the list, the cumulative 

transparency is: 

CT(n) = (CT(n-l) * (D(n) * op(n) + E(n)) + C(n) * op(n)) 
* A(n) EQUATION 3 

Where "n" is the current level and CT(n) is the cumulative 

15 transparency for the current level. The bit counts are sent 
to the cumulative transparency calculator 130 over a line 
147 (FIGURE 10A) - 

The cumulative transparency is the sum of two terms , 
one due to current sub-pixels chat are not behind any other 

20 polygon, and the other due to current sub-pixels that are 
hidden by prior polygons. The first term is the prior level 
transparency (CT(n-l)J times the sum of the number of sub- 
pixel bits (D(n)) that are used by both the current level 
polygon and at least one prior polygon times the current 

25 polygon's opacity (op(n)) plus the number of sub-pixel bits 
used only by the prior polygons (E(n)). The second term is 
the current polygon's opacity times the number of sub-pixel 
bits used only by the current polygon (C(n)). These terms 
are weighted by their respective number of sub-pixels, added 

30 together, then divided by the total number of active sub- 
pixels at the current level (A(n)) to normalize the result. 

After the current level's cumulative transparency is 
calculated, the cumulative shade calculator 138 generates 
the cumulative pixel shade starting at a block 116 ( FIGURE 

35 HA) by retrieving the cumulative shade generated! for the 

prior level. At a block 118, as each polygon in the list is 
processed, the pixel shader 3 6 finds the nearest prior level 
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in the list that has sables that conflict with the polygon 
being processed . This is done by comparing the current 
polygon sub-pixel mask with each of the prior cumulative 
reaS ks and identifying the closest level that has bits in 
5 common with the current level. 

At a block 120. the cumulative shade calculator 13B 
combines the current polygon's shade with the prior 
cumulative shade. The current polygon's sub-pixel bits *re 
divided into two sets: those that overlap bits in the 
10 nearest prior cumulative level and those that do not (and 
hence are not obscured by any prior polygon) . Unobscured 
sub-pixels are counted, and their contribution to the 
display video is the current polygon color (color (n)) times 
its opacity (op(n)) times this count (C(n)) ( EQUATION 5). 
15 The overlapping sub-pixel bits are counted, and their 

contribution to the display video is equal to the currer.t 
polygon color times its opacity times the cumulative 
transparency (CT(B» of the nearest prior conflicting level 
(B) times the overlap count (D(n)). The current polygons'* 
20 color and opacity are accessed over the line 146. The hxt 
counts are sent to the cumulative shade calculator 138 over 
the line 147 (FIGURE 10A) - The cumulative transparency of 
the nearest conflicting level is accessed over a line 148. 

For the first polygon in the list, the shade at level o 

25 (S(0)) isi 

6(0) - op(0) * color(O) * A(0) EQUATION 4 

The cumulative shade is generated for each polygon that 
influences the pixel being processed (block 122). For 
polygons at levels other than level 0 in the list, the 
30 cumulative shade (S(n)) is: 

S(n) = S(n-l) + (C(n) + D(n) * CT(B)) + op(n) * color(n) 

EQUATION 5 

After all the polygons have been processed, the pixel 
shader 36 generates the final pixel shade and outputs it 
35 over a line 149 (block 124) . The final pixel shade 

(S (final)) is equal to the final cumulative shade divided by 
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the number of sub-pixels: 

S (final) S(last) -i- A(last) EQUATION 6 

Where S(last) is the cumulative shade at the last level and 
A (last) is the number of sub-pixels used by all the levels . 
5 Typically, A (last) is equal to the total number of sub- 
pixels in a pixel (e.g., 16). However, in some applications 
(e.g., workstations), A(last) will be less than the totaL 
number of sub-pixels. The pixel shader process terminates 
at a block 126, 

10 An alternative embodiment of the pixel shader 3 6 is 

depicted in FIGURE 10B. As in the embodiment of FIGURE ZOh, 
cumulative transparency data and cumulative shade data are 
generated as the polygons are processed in depth sort order* 
However, the cumulative shade is generated from a cumulative 

15 combined record 150 and a cumulative resolved record 151. 
The cumulative records are blended based on sub-pixel 
conflicts detected by a conflict checker 152 and an overlap 
calculator 153. As above, the pixel shader 36, uses 
cumulative transparency values generated by a cumulative 

20 transparency calculator 154 to calculate how much of the 

current polygon should be affected by prior transparency and 
how much should show through unaltered - 

The operation of the pixel shader of FIGURE 10B is 
illustrated in FIGURE 11B starting at a block 260. 

25 Initially, the combined record and resolved record storage 
is cleared before any polygons are processed. The color 
values for the polygon are multiplied by the opacity of the 
polygon before they are passed to the section of the pixel 
shader 36 that performs the final transparency calculations. 

3 0 For example, a white polygon with 25% opacity would have 
color values (red, green, blue and alpha) of 0.25. 

At a block 262, polygon data for the current polygon is 
provided on a line 155 (FIGURE 10B) . Conflict checker 152 
accesses the polygon's coverage mask bits over a line 156 

35 and compares them with mask bits retrieved from the 

aggregate combined record 150 over a line 157 (block 264). 
At a block 266, if there are no coverage mask bits that 
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conflict, the process proceeds to a block 268 where the 
■ conflict checker 152 sends a signal to the combined record 
150 over the line 157 indicating that the polygon is to be 
blended with the aggregate combined record. 
5 When a polygon is blended into the combined record 150. 

the contribution of the polygon is weighted by the number of 
bits set in the polygon's coverage mask. For example, the 
new combined record red color data (CbRecRed) is: 

CbRecRed = CbRecRed + PolyRed * SetBits (PolyMask) 

EQUATION 7 

Where the CbRecRed to the right of the equal sign is the 
prior combined record red color, PolyRed is the polygons 
red color data value and SetBits ( Pol yMask) is the number of 
bits set in the polygon's coverage mask (PolyMask). Similar 
15 equations are used for the green, blue and alpha components 
of the color information. 

The new combined record transparency (CbRecCumTran) is: 
cbRecCumTran = CbRecCumTran + PolyTran * SetBits (PolyMask) 

EQUATION 8 

20 Where the CbRecCumTran to the right of the equal sign is the 
prior combined record cumulative transparency. PolyTran is 
the polygon's transparency which is equal to 1 - "the 
polygon's opacity value." 

Finally, the new combined record mask (CbRecMask) is 
25 set to the logical "or" of the polygon's coverage mask and 
the combined record's current mask. 

If any of the same mask bits were set in the two masks 
at block 266. the process proceeds to a block 272 instead of 
block 268. The conflict checker 152 sends a signal to the 
30 resolved record 151 over a line 158 indicating that the 
aggregate combined record 150 is to be blended into the 
resolved pixel record 151. 

When a combined record is added to the resolved record, 
the contribution of the combined record is weighted by the 
35 number of bits set in the combined record's cumulative *ask. 
The contribution of the combined record for those sub-pixels 
that are in conflict is reduced based on the current 
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cumulative transparency stored in resolved record 
(RsvReeCuinTran) . At block 272, the overlap calculator 
generates bit counts using the combined record mask and 
resolved record mask (RsvRecMask) which are accessed over 
5 lines 159 and 160, respectively. These bit counts 

(Conflict, UncnOld and UncnNew) are sent to the resolved 
record and the cumulative transparency calculator over lines 
160 and 161, respectively. These bit counts are calculated 
as follows: 

10 Conflict = SetBits (RsvRecMask AND CbRecMask) 

UncnOld setBits (RsvRecMask AND (NOT CbRecKask) ) 
UncnNew « SetBits ((NOT RsvRecMask) AND CbRecMask) 

EQUATION 9 

Where "AND" and "NOT" are the corresponding logic 

15 operations. 

At a block 274, the pixel shader 36 fetches the 
cumulative transparencies for the combined record and the. 
resolved record over lines 162 and 163 , respectively- These 
transparencies were Btored the last time the combined record 

20 and the resolved record were blended. 

At a block 276, the combined record shade (accessed 
over a line 164) is blended with the resolved record shade. 
Using the red color data as an example, the new resolved 
record red color data (RsvRecRed) is: 

25 RsvRecRed = RsvRecRed + (CbRecRed * setBits (CbRecMask) ) * 

Conflict * RsvReccumlTran + (CbRecRed * 
SetBits {CbRecMask) ) * UncnNew EQUATION 10 

Again, similar equations are used for the green, blue and 
alpha components. 

30 The new value of the cumulative transparency for the 

resolved record calculated at a block 278 is the weighted 
sum of three terms: (l) the cumulative transparency of 
resolved record (RsvRecCumlTran) weighted by the number of 
nonconf licting old sub-pixels (UncnOld); (2) the product af 

35 the transparency of resolved record (RsvRecCumlTran) and the 
transparency of combined record (cbRecCumlTran) weighted oy 
the number of conflicting sub-pixels between the two records 
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(conflict); and (3) the cumulative transparency of combined 
" record (CbRecCumlTran) weighted by the number of 
nonconf lie ting new sub-pixels (UncnNew) : 

RsvRecCumlTran - (RsvRecCumlTran * UncnOld + 

RsvRecCumlTran * (CbRecCumlTran + setBits (CbRecMask) ) 
* conflict + (CbRecCumlTran * SetBits (CbRecMask) ) * 
UncnNew) + (UncnOld + Conflict + UncnNew) 

EQUATION 11 

The new resolved record mask (RsvRecMask) is set to the 
logical -or" of the combined record's mask (CbRec*ask) and 
the resolved record's current mask. 

Finally, at a block 280, the conflict checker 152 sends 
a signal to the combined record 150 over the line 157 which 
causes the incoming polygon to be written as the new 
IS aggregate combined record. 

&t a block 270, the process returns to block 262 iJ; 
there are more polygons to be processed for this pixel . If 
the last polygon has been processed for this pixel, the 
pixel Shader 36 computes the final shade at a block 282. 
20 After all polygons have been processed, any data remaining 
in combined record 150 is combined into resolved record 151 
using the same methods that would be used if another polygon 
were processed which conflicted with the data in combined 
record 150. In one embodiment, this is done by setting a 
25 "dummy" polygon equal to the current combined record. The 
dummy polygon is sent to the pixel shader causing the 
combined record data to be blended with resoled record as 
desired. The dummy polygon will be cleared when the next 
pixel shader process commences. 
30 The final pixel shade is equal to the final resolved 

record value divided by the number of active sub-pixels - 
Again, using the red color data as an example: 

FinalRed = RSvRecRed + NoSubPix EQUATION 12 

Where NoSubPix is the number of active sub-pixels, e.g., 16. 
35 The final pixel shade is output over a line 165 and th«> 
process terminates at a block 284. 



05/08/2003 12:05 FAX 404 873 8501 



ARNALL GOLDEN & GREGORY 



©064 



WO 97M1536 PCT/US97/07447 

28 

The above algorithms have an important characteristic. 
Within a tiling of transparent polygons, such as might bo 
used to construct a windshield or aircraft canopy, polygons 
can't affect each other. That is, the transparency of prior 
5 polygons within the tiling won't be applied to subsequent 
polygons within the tiling- Since their 6Ub-pixel masks are 
mutually exclusive, they won't inadvertently "double^cover" 
each other with a double dose of transparency. in addit.ion, 
by calculating the transparency of a combined record of a 
10 tiling of multiple polygons, the correct transparency for 
that tiling will be applied to subsequent polygons behind 
the tiling. 

To support immediate -mode graphics, the entire process 
is repeated every time a polygon is rendered, for every 

15 pixel the polygon influences- The display video that slowly 
accumulates during a rendering frame is always available to 
the video processor. 

The procedure set forth above provides an efficient 
method of blending polygons, in comparison, a rigorously 

2 0 correct approach for blending transparent polygons would 
require a depth-sort order for every sub-pixel. For 
example, 16 differently sorted lists would be required fcr 
every pixel in a system that uses 16 sub-pixels per pixel . 
The polygons in each sub-pixel list would be processed in 

25 the listed order to develop a final sub-pixel color. Then, 
all the sub-pixel colors would be added together to generate 
the final shade for the pixel. This approach is not very 
cost-effective. In contrast, the disclosed procedure is 
much simpler yet yields almost identical results. 

30 Once the final pixel shade is computed, it is stored in 

the frame buffer 26 over a line 96. when an image frame is 
to be displayed, the final pixel shades are read from the 
frame buffer 26 and sent to the display device 20 over a 
line 96- The display device 20 converts the pixel shade 

35 data to electrical signals which illuminate the pixels on 
the display screen. 
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The display device 20 is implement* using a pixel- 
based display. Techniques for scanning frame buffers to 
drive displays pixel-by- P ixel are well known in the art^ 
F or example, various formats for organizing and — ^ 
5 frame buffers to drive displays pixel-by-pix*l are discussed 
in the textbook r o ster dj nlnrn- rrjnH r^ ^ iWtn *. 
2nd Edition, Foley, van Dam, Feiner * Hughes, 1991. by 
Addison-Wesley Publishing company, at chapters 4 and 18- 
There is a critical difference between the above 
10 process and conventional aultisampler implementations . «hen 
the frame buffer is organic as a per-polygon. per-sample 
structure, a sample can be claimed by only one polygon In 
general, order- independent transparency effects can only ba 
achieved with "screen-door" strategies and multiple overlap 
15 transparent polygons cannot be made to combine properly. 
Also, edges that are behind transparent polygons are 
rendered with substandard image quality, in contrast, by 
organizing the frame buffer on a per-polygon, per-pixel 
basis and storing a full set of sub-pixel bits with each 
20 polygon, proper composite effects can still be 

when transparent polygons overlap other polygons in a p.xel. 
The system is no longer limited to just 16 levels of 
screen-door transparency. Instead, it can have as many as 
the selected data format for opacity allows. Typically 8 or 
2 S 12 bits are used. Thus, 256 or 4096 levels would be 

available. Furthermore, edges behind such transparence 
are rendered with full 16-multisample quality. 

The images generated by the disclosed embodiment are of 
very high quality in a. number of respects. For example, the 
quality of edges that are behind transparent polygons is 
typically as good as the quality of the other edges in the 
scene. In addition, edges of intersection where polygons 
pierce one another are typically displayed with relatively 
high quality. Finally, the occupation solver 32 
effectively resolves difficult depth problems even without 
algorithmic or numerical assists being given to the solver. 



30 

scene 

35 
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The disclosed embodiment generally is implemented using 
standard computer graphics system components and 
incorporating some form of graphics pipeline. Typically r 
this pipeline uses an image generator that consists of a 
5 central processor unit and graphics processor, the basic 
concepts of which are disclosed in the book Fundamental^ 
Interactive Computer Gfapftiflp , Foley and Van Dam r 1984 , 
Addison-Wesley Publishing Company, at chapters 4 and 18. 
Depending on system requirements, the sampling engine 23, 
10 occultation solver 32, depth sorter 34 and pixel shader 36 
operations described above may be performed on the same 
processor or different processors* 

The details of polygon processing and the corresponding 
structures used to implement these processes are also vejII 
15 known in the computer graphics art. Several of these 

techniques and structures are discussed at length in the 
books Principles of Interactive r oniputgr Graphics . 2nd 
Edition, Newman and Sproull, McGraw-Hill Book Company, 3 979, 
and Computer Graphics! Prinpip] P „g anri Practice , 2nd Edition, 
20 Foley, van Dam, Feiner & Hughes, Addison-Wesley Publishing 
Company, 1991. 

The polygon memory 22, the polygon buffer 30 and the 
frame buffer 26 typically are implemented using a 
conventional RAM data memory. Nevertheless, these 
25 components may be implemented using any suitable data 

storage method. In addition, the polygon buffer 30 may be 
incorporated into the image generator or other processors 
depending on the selected system design. 

Some of the above functions may be implemented using 
3 0 functionally equivalent components including, but not 

limited to, microprocessors, custom microchips, discrete 
components and the like. Furthermore, significant portions 
of the above operations typically would be implemented using 
computer software programs. The details of these and 
35 related implementations are well known in the art of 
computer systems and computer graphics systems- 
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Finally, the lines in FIGURE 1 (e.g., lines 54 and 96) 
' generally represent the flow of data from one operation to 
another. Thus, these lines may be implemented using any 
number of data flow techniques including, but not limited 
5 to, data busses that connect the data ports of discrete 
components or busses that are located inside integrated 
components. In addition, in integrated computer graphics 
systems, the flow of data from one component block, to 
another might be implemented using computer program 
10 parameter passing operations, inter-process communications 
or other software techniques. 

With the structure and function of the components of 
the present invention in mind, the operation of processing a 
polygon to generate pixel shade data as performed by tho 
15 embodiment of FIGURE 1 is treated in FIGURES 9A and 9B 
starting at a blocX 200 (FIGURE 9A, top) . 

Initially, the sampling engine 28 (FIGURE 1) determines 
which pixel sub-pixels are influenced by the polygon. At a 
block 202. the sampling engine 28 retrieves a polygon from 
20 the polygon memory 22. Then, at a block 204, the sampling 
engine 28 generates the sub-pixel mask for a pixel 
influenced by the polygon. A polygon/pixel fragment is 
output along with the fragment's shading and edge-mask sub- 
pixel data and the indices of the pixel it is to be comoined 
25 into. 

If no prior polygons have been processed for this pixel 
(block 206), the new. polygon data is stored in the 
pre-allocated memory location at a block 208 and processing 
proceeds to the shader stage. 

3 0 If there are prior polygons, the loop represented by 

blocks 210 through 218 is executed for each one. At block 
210, polygon data for a prior polygon is retrieved from the 
frame buffer 26 and stored in temporary buffers (e.g.. 
polygon buffer 30) in the multisampler 24. 

35 The occultation solver 32 computes modified sub-pixel 

bits for the prior and new polygons (block 212). At a block 
214, if the new polygon loses all its samples, processing on 
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this polygon/pixel fragment ceases, in this case, no data 
is changed in the frame buffer 26 and the process proceeds 
to block 218. Occultation processing may also be skipped if 
there is no overlap between the new and prior polygons- sub- 
pixel bits or if both polygons are transparent. 

If the new polygon retains at least one sample, thu 
process proceeds to block 216 were the pixel center depth 
value is used to determine whether the new polygon is cj.oser 
or farther than the prior polygon, if the new polygon is 
closer, the new polygon is inserted into the list of prior 
polygons at this buffer location. All other prior polygons 
will slide down one slot. If the new polygon is farther, 
the list remains unchanged, if all prior polygons are in 
front of the new polygon, it is appended to the end of the 
15 temporary buffer list. 

At block 218, if there are wore prior polygons in the 
list, the loop returns back to block 210. After all prior 
polygons have been processed, the remaining sub-pixel bits 
of the new polygon are updated at its buffer location (block 
20 220) . 

At this point, all prior polygons have been read from 
the frame buffer 26 into temporary buffers and the new 
polygon has been inserted into its proper place in the list. 
The new polygon's occultation interactions with all prior 

25 polygons has been computed and appropriate changes have been 
made in the sub-pixel mask bits. It should be noted that 
since buffer allocation is based on pixel center depth, ;Lt 
is possible for more distant polygons to still occult soiae 
sub-pixels of the new polygon, 

30 Next, the shader/ storage loop represented by blocks 222 

through 23 6 is performed for each polygon in the list. At 
block 222, a polygon is selected from the list. 

At a block 224, if a polygon buffer entry no longer has 
active sub-pixels, the process proceeds to a block 228. If 

35 the polygon buffer entry does have active sub-pixels, ite. 

effect is added to the accumulating pixel shade based on its 
color, opacity, number of sub-pixel bits, and the 
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transparency of any prior polygons that overlap any sub- 
pixels with it (block 226) . 

Next, the polygon buffer data is stored in the frame 
buffer 26. At a block 228, if a new heap location is 
5 required it is allocated (block 230) and the appropriate 
heap pointers are properly chained (block 232) . At a block 
234, the polygons are read from the temporary buffers ard 
written to the frame buffer 26. Polygons whose sub-pixel 
bits were all cleared by the occulta tion solver 32 are not 
10 used, nor are they stored back in the frame buffer 26. 

At a block 236, if more polygons in the list need to be 
processed, the process return back to block 222- At th* end 
of the shader loop, the computed shade is stored in the 
pixel video location (block 238) . 
15 At a block 240, if any more pixels that are influenced 

by the new polygon need to be processed, the process returns 
to block 204 where the above steps are performed for the! 
next pixel. Otherwise the process terminates at a block 
242. 

20 From the above, it is apparent that the system 

disclosed herein utilizing adaptive pixel multisampling 
offers an improved system for generating pixel data. 
Recognizing that the system can be implemented with standard 
graphics and display components, it should be noted that 

25 considerable variation may occur in the specific components 
and operating format. The scope of the present invention 
should be determined with a reference to the claims set 
forth below. 
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WHAT IS CLAIMED IS: 

1 l. A computer graphics system for generating pixul 

2 data to display images on a display device utilizing an 

3 array of pixels, comprising: 

4 a data memory for storing primitives 

5 representative of said images; 

6 a frame buffer for storing said pixel data 

7 including, means for storing a plurality of primitive 

8 fragments for a pixel wherein each of said fragments is 

9 associated with a distinct primitive; and 

10 a processor for processing said primitives to 

11 generate said primitive fragments wherein said 

12 processor stores a plurality of said primitive 

13 fragments for a pixel in said frame buffer, said 

14 processor for processing said primitive fragments to 

15 generate composite pixel data to display images on said 

16 display device* 

1 2. The computer graphics system of claim l further 

2 including a depth sorter for depth ordering said primitive 

3 fragments for a pixel. 

1 3. The computer graphics system of claim 2 wherein 

2 said depth sorter orders said primitive fragments using 

3 pixel-center depth values. 

1 4. The computer graphics system of claim 1 further 

2 including a pixel shader for combining said primitive 

3 fragments according to sub-pixel use by a current polygon 

4 and at least one prior polygon. 

1 5. The computer graphics system of claim 4 further 

2 including an occultation solver wherein said primitive 

3 fragments include sub-pixel information and said occultation 

4 solver modifies said sub-pixel information to eliminate 

5 hidden primitives. 
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6, The computer graphics system of claim 1 furthor 
including a sampling engine for generating sub-pixels ui;ing 



3 non-contiguous sub-pixel areas. 



7. The computer graphics system of claim 6 wherein 
said sub-pixel areas interlock so that every portion af an 
image plane defined by said pixels is associated with only 



4 one of said pixels. 



! 8- The computer graphics system of claim 1 wherein at 

2 least a portion of said frame buffer is managed as a he.*p- 



9, The computer graphics system of claim 1 further 
including a display device containing an array of pixels for 



3 displaying said images. 



10, The computer graphics system of claim l further 
including a pixel shader for combining said primitive 
fragments wherein said pixel shader includes a cumulative 

4 transparency generator for calculating a cumulative 

5 transparency for a tiling of polygons in a combined record. 

1 11. The computer graphics system of claim 10 further 

2 including a resolved record for storing cumulative combined 

3 record data. 

1 12. A computer graphics process for generating pixel 

2 data to display images on a display device containing an 

3 array of pixels, comprising the steps of: 

4 storing primitives representative of said images; 

5 processing said primitives to generate a plurality 

6 of primitive fragments for a pixel wherein each of said 

7 fragments is associated with a distinct primitive; 

8 storing said plurality of primitive fragments for 

9 a pixel in a memory; and 
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processing said primitive? fragments to generate 

11 composite pixel data to display images on said display 

12 device. 

1 13. The computer graphics process of claim 12 further 

2 including the step of depth ordering said primitive 

3 fragments for a pixel. 

1 14- The computer graphics process of claim 13 wherein 

2 said depth ordering includes the step of ordering said 

3 primitive fragments using a pixel-center depth value. 

1 15. The computer graphics process of claim 12 further 

2 including the step of processing said primitive fragments 

3 according to sub-pixel use by a current polygon and at least 

4 one prior polygon - 

1 16. The computer graphics process of claim 12 further 

2 including the step of processing said primitive fragments by 

3 generating a cumulative transparency for a tiling of said 

4 primitives in a combined record. 

1 17. The computer graphics process of claim 16 furtJier 

2 including the step of generating a resolved record from 

3 cumulative combined record data. 

1 18, The computer graphics process of claim 12 further 

2 including the step of generating sub-pixels using non- 

3 contiguous sub-pixel areas, 

1 .19- The computer graphics process of claim 18 wherein 

2 said sub-pixel areas interlock so that every portion of an 

3 image plane defined by said pixels is associated with onj.y 

4 one of said pixels. 
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20. The computer graphics process of claim 12 further 
' including the step of managing at least a portion of said 



3 memory as a heap, 



21. A computer graphics process for generating pix«l 
data to display images on a display device containing an 
3 array of pixels, comprising the steps oft 

storing primitives representative of said images; 
defining non-contiguous sub-pixel areas for said 

6 pixels; 

7 processing said primitives using said sub-pixel 
areas to compute sub-pixel pixel data; and 

processing said sub-pixel pixel data to generate 
composite pixel data to display images on said display 



IX device. 



22. The computer graphics process of claim 21 wherein 
said sub-pixel areas for one of said pixels do not overlap 
any of said sub-pixel areas for another of said pixels. 



23. The computer graphics process of claim 22 wherein 
said sub-pixel areas for one of said pixels interlocks vith 



1 
2 

3 said sub-pixel areas for another of said pixels 



24. The computer graphics process of claim 21 further 
including the step of storing, for each of said pixels, said 
sub-pixel pixel data for each of said primitives that 



4 contribute to said each of said pixels. 
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